\documentclass[11pt]{book}

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%                                                                                                 %
% The mathematical style of these documents follows                                               %
%                                                                                                 %
% A. Thompson and B.N. Taylor. The NIST Guide for the Use of the International System of Units.   %
%    NIST Special Publication 881, 2008.                                                          %
%                                                                                                 %
% http://www.nist.gov/pml/pubs/sp811/index.cfm                                                    %
%                                                                                                 %
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

\input{../Bibliography/commoncommands}
\IfFileExists{../Bibliography/gitrevision.tex}
{\input{../Bibliography/gitrevision}}
{\newcommand{\gitrevision}{unknown} }

\externaldocument[UG-]{FDS_User_Guide}

\includeonly{Overview_Chapter,Survey_Chapter,Experiment_Chapter, Error_Chapter, HGL_Chapter, Plume_Chapter, Ceiling_Jet_Chapter,Velocity_Chapter, Species_Chapter,Pressure_Chapter, Surface_Temperature_Chapter, Heat_Flux_Chapter, Suppression_Chapter, Burning_Rate_Chapter, Wind_Chapter}

%\includeonly{Experiment_Chapter,Suppression_Chapter}

\begin{document}

\bibliographystyle{unsrt}
\pagestyle{empty}

\begin{minipage}[t][9in][s]{6.5in}

\headerA{
1018-3\\
Sixth Edition\\
}

\headerB{
Fire Dynamics Simulator\\
Technical Reference Guide\\
Volume 3: Validation\\
}

\headerC{
\authortitlesigs
}

\vfill

\headerD{1018}

\vfill

\logosigs

\end{minipage}

\newpage

\hspace{5in}

\newpage

\begin{minipage}[t][9in][s]{6.5in}

\headerA{
1018-3\\
Sixth Edition\\
}

\headerB{
Fire Dynamics Simulator\\
Technical Reference Guide\\
Volume 3: Validation\\
}

\headerC{
\authorsigs
}

\headerD{1018}

\headerC{
\flushright{\today \\
Revision:~\gitrevision}}


\vfill

\flushright{\includegraphics[width=1in]{../Bibliography/doc} }

\titlesigs

\end{minipage}

\newpage

\disclaimer{1018-3}




\newpage

\frontmatter

\pagestyle{plain}

\input{../Bibliography/authors}


\chapter{Preface}

This is Volume 3 of the FDS Technical Reference Guide. Volume 1 describes the mathematical model and numerical method. Volume 2 documents past and present model verification work. Instructions for using FDS are contained in a separate User's Guide~\cite{FDS_Users_Guide}.

The FDS Technical Reference Guide is based in part on the ``Standard Guide for Evaluating the Predictive Capability of Deterministic Fire Models,'' ASTM~E~1355~\cite{ASTM:E1355}. ASTM~E~1355 defines {\em model evaluation} as ``the process of quantifying the accuracy of chosen results from a model when applied for a specific use.'' The model evaluation process consists of two main components: verification and validation. {\em Verification} is a process to check the correctness of the solution of the governing equations. Verification does not imply that the governing equations are appropriate; only that the equations are being solved correctly. {\em Validation} is a process to determine the appropriateness of the governing equations as a mathematical model of the physical phenomena of interest. Typically, validation involves comparing model results with experimental measurement. Differences that cannot be explained in terms of numerical errors in the model or uncertainty in the measurements are attributed to the assumptions and simplifications of the physical model.

Evaluation is critical to establishing both the acceptable uses and limitations of a model. Throughout its development, FDS has undergone various forms of evaluation, both at NIST and beyond. This volume provides a survey of validation work conducted to date to evaluate FDS.

\input{../Bibliography/disclaimer}


\chapter{Acknowledgments}
\label{acksection}

The following individuals and organizations played a role in the validation process of FDS.
\begin{itemize}
\item The US Nuclear Regulatory Commission Office of Research has funded key validation experiments, the preparation of the FDS manuals, and the development of various sub-models that are of importance in the area of nuclear power plant safety. Special thanks to Mark Salley, David Stroup, and Jason Dreisbach for their efforts and support.
\item Anthony Hamins of NIST directed the NIST/NRC and WTC experiments, conducted smaller methane burner measurements, and quantified the experimental uncertainty of these and other experiments used in this study. Alex Maranghides was the Director of the Large Fire Laboratory at NIST at the time these tests were conducted, and he helped to design the experiments. Therese McAllister oversaw the instrumentation of the structural steel during the WTC experiments.
\item Anthony Hamins of NIST developed the technique of evaluating experimental uncertainty that is used throughout this Guide. Blaza Toman of the Statistical Engineering Division of NIST developed the method of quantifying the model uncertainty.
\item Rick Peacock of NIST assisted in the interpretation of results from the ``NBS Multi-Room Test Series,'' a set of three room fire experiments conducted at the National Bureau of Standards (now NIST) in the mid-1980's.
\item Bryan Klein, currently employed at Thunderhead Engineering, Inc., assisted in the development of techniques to automatically generate the plots that are found throughout this Guide.
\item Bill Pitts, Nelson Bryner, and Erik Johnsson of NIST contributed and interpreted test data for the ``NIST Reduced Scale Enclosure Experiments.'' Matthew Bundy, Erik Johnsson, Paul Fuss, David Lenhart, Sung Chan Kim, and Andrew Lock of NIST contributed similar data collected within a full-scale standard compartment in 2010.
\item Rodney Bryant of NIST contributed velocity profile data for the ``Bryant Doorway'' series.
\item Tony Putorti and Scott Bareham of NIST contributed temperature measurements from plate thermometer experiments in a cone calorimeter.
\item David Sheppard, currently of the Bureau of Alcohol, Tobacco and Firearms (ATF), conducted the experiments referred to as the ``UL/NFPRF Test Series'' on behalf of the Fire Protection Research Foundation (then known as the National Fire Protection Research Foundation) while working at Underwriters Labs in Northbrook, Illinois. Sheppard, along with Bryan Klein, currently employed at Thunderhead Engineering, Inc., conducted the experiments referred to as the ``ATF Corridors'' series in 2008.
\item Jerry Back, Craig Beyler and Phil DiNenno of Hughes Associates and Pat Tatem of the Naval Research Laboratory contributed experimental data for the ``HAI/NRL Wall Fire'' series. Thanks also to Craig Beyler for assistance with the data for the ``Beyler Hood Experiments.''
\item Ken Steckler provided details about the ``Steckler Compartment Experiments'' of 1979.
\item Jianping Zhang at the University of Ulster contributed heat flux measurements from the SBI apparatus.
\item At the University of Maryland, Professor Fred~Mowrer and Phil Friday were the first to apply FDS to the NRC-sponsored experiments referred to in this document as the ``FM/SNL Test Series'' (Factory Mutual and Sandia National Laboratories conducted these experiments).
\item Jukka Vaari of VTT, Finland, contributed the Cup Burner test cases.
\item Steve Nowlen of Sandia National Laboratory provided valuable information about the FM/SNL series, and he also conducted the CAROLFIRE experiments.
\item Ulf Wickstr\"{o}m of SP, Sweden, contributed experimental data from a series of experiments (SP AST) that were designed to evaluate the feasibility of using plate thermometer measurements as boundary conditions for a heat conduction calculation within several types of steel beams. The adiabatic surface temperature concept was tested in both the experiments and model.
\item Jeremy Thornock at the University of Utah provided data on the Sandia helium plume.
\item Sheldon Teiszen at Sandia National Laboratories, Albuquerque, provided detailed statistics for the helium plume and pool fire experiments conducted in the Sandia FLAME facility.
\item Taylor Myers, a student at the University of Maryland and a Summer Undergraduate Research Fellow (SURF) at NIST, analyzed the Vettori Flat and Sloped Ceiling sprinkler experiments. Thanks also to Bob Vettori of the U.S. Nuclear Regulatory Commission and formerly of NIST for his help in locating the original test data and laboratory notebooks.
\item Hans la Cour-Harbo, a masters degree student at the Technical University of Denmark, provided guidance and insight on the NRCC Facade experiments. Scott Bareham of NIST provided the technical drawing of the test enclosure.
\item Prof.~Stanislav Stoliarov and graduate students Mark McKinnon and Jing Li of the University of Maryland provided the properties of several polymers for the FAA Polymers example.
\item Michael Spearpoint and masters degree students Roger Harrison and Rob Fleury of the University of Canterbury, New Zealand, supplied measurements of mass entrainment rates into spill plumes (Harrison Spill Plumes) and heat flux measurements from propane burner fires (Fleury Heat Flux).
\item Ezti Oztekin of the Federal Aviation Administration (FAA) developed the FAA Cargo Compartments cases based on experiments sponsored by the FAA.
\item Topi Sikanen of VTT, Finland, and Jonathan Wahlqvist of Lund University, Sweden, contributed FDS input files for the PRISME DOOR series.
\item Paul Tyson, a student at the University of Ulster, Northern Ireland, contributed the input files and supporting documents for the NRCC Smoke Tower experiments.
%\item The Wind Engineering chapter was developed with help from Scott Hemley (2012 NIST SURF student), Dilip Banerjee, Donghun Yeo, Marc Levitan, and Emil Simiu, all from the NIST Engineering Laboratory, Materials and Structural Systems Division.
\item James White, a student at the University of Maryland, provided documentation and input files for the UMD Line Burner cases.
\item Charlie Hopkin and Michael Spearpoint of Olsson Fire \& Risk provided the data and FDS input files for experiments conducted by Adam Bittern at the University of Christchurch, New Zealand.
\item The simulations of liquefied natural gas (LNG) dispersion experiments that are described in this report were originally designed by Jeffrey Engerer and Anay Luketa of Sandia National Laboratories on behalf of the Pipeline and Hazardous Materials Safety Administration of the U.S. Department of Transportation.
\end{itemize}


\cleardoublepage
\phantomsection
\addcontentsline{toc}{chapter}{Contents}
\tableofcontents

\cleardoublepage
\phantomsection
\addcontentsline{toc}{chapter}{List of Figures}
\listoffigures

\cleardoublepage
\phantomsection
\addcontentsline{toc}{chapter}{List of Tables}
\listoftables

\chapter{List of Acronyms}

\begin{tabbing}
\hspace{1.5in} \= \\
ALOFT \> A Large Outdoor Fire plume Trajectory model \\
AST \> Adiabatic Surface Temperature \\
ASTM \> American Society for Testing and Materials \\
ATF \> Bureau of Alcohol, Tobacco, Firearms, and Explosives \\
BRE \> British Research Establishment \\
CAROLFIRE \> Cable Response to Live Fire Test Program \\
CFAST \> Consolidated Model of Fire Growth and Smoke Transport \\
CHRISTIFIRE \> Cable Heat Release, Ignition, and Spread in Fire Test Program \\
CFT \> Critical Flame Temperature \\
DNS \> Direct Numerical Simulation \\
FAA \> Federal Aviation Administration \\
FDS \> Fire Dynamics Simulator \\
FLAME \> Fire Laboratory for Accreditation of Models by Experimentation \\
FM \> Factory Mutual Global \\
FSE \>  Full-Scale Enclosure \\
HAI \> Hughes Associates, Inc. \\
HDPE \> high density polyethylene \\
HGL \> Hot Gas Layer \\
HIPS \> high-impact polystyrene \\
HRR \> Heat Release Rate \\
ISO \> International Standards Organization \\
LEMTA \> Laboratoire d'Energ\a'{e}tique et de M\a'{e}chanique Th\a'{e}orique et Appliqu\a'{e}e \\
LES \> Large Eddy Simulation \\
LLNL \> Lawrence Livermore National Laboratory \\
LNG \> Liquified Natural Gas \\
MEC \> Minimum Extinguishing Concentration \\
NBS \> National Bureau of Standards (former name of NIST) \\
NFPRF \> National Fire Protection Research Foundation \\
NIST \> National Institute of Standards and Technology \\
NRC \> Nuclear Regulatory Commission \\
NRCC \> National Research Council of Canada \\
NRL \> Naval Research Laboratory \\
PDPA \> Phase Doppler Particle Analyzer \\
PIV \> Particle Image Velocimetry \\
PMMA \> poly(methyl methacrylate) \\
PRISME \> Propagation d'un incendie pour des sc\a'{e}narios multi-locaux \a'{e}l\a'{e}mentaires \\
PVC \> Polyvinyl chloride \\
RANS \> Reynolds Averaged Navier-Stokes \\
RSE \> Reduced-Scale Enclosure \\
SBI \>  Single Burning Item \\
SNL \> Sandia National Laboratory \\
SP \>  Statens Provningsanstalt (Technical Research Institute of Sweden) \\
TGA \> Thermal Gravimetric Analysis \\
THIEF \> Thermally-Induced Electrical Failure \\
UL  \> Underwriters Laboratories \\
USN \> United States Navy \\
VTFRL \> Virginia Tech Fire Research Laboratory \\
VTT \> Valtion Teknillinen Tutkimuskeskus (Technical Research Centre of Finland) \\
WTC \> World Trade Center \\
\end{tabbing}



\mainmatter

\include{Overview_Chapter}

\include{Survey_Chapter}

\include{Experiment_Chapter}

\include{Error_Chapter}

\include{HGL_Chapter}

\include{Plume_Chapter}

\include{Ceiling_Jet_Chapter}

\include{Velocity_Chapter}

\include{Species_Chapter}

\include{Pressure_Chapter}

\include{Surface_Temperature_Chapter}

\include{Heat_Flux_Chapter}

\include{Suppression_Chapter}

\include{Burning_Rate_Chapter}

\include{Wind_Chapter}


\chapter{Conclusion}


\section{Summary of FDS Model Uncertainty Statistics}

Table~\ref{summary_stats} lists the summary statistics for the different quantities examined in this Guide. This is, for each quantity of interest, Table~\ref{summary_stats} lists the bias and relative standard deviation of the predicted values. It also lists the total number of experimental data sets on which these statistics are based, as well as the total number of point to point comparisons. Obviously, the more data sets and the more points, the more reliable the statistics.

For further details about model uncertainty and the meaning of these statistics, see Chapter~\ref{Error_Chapter}.

\IfFileExists{SCRIPT_FIGURES/ScatterPlots/validation_statistics.tex}{\input{SCRIPT_FIGURES/ScatterPlots/validation_statistics.tex}}{\typeout{Error: Missing file SCRIPT_FIGURES/ScatterPlots/validation_statistics.tex}}


\section{Normality Tests}
\label{normality_tests}

The histograms on the following pages display the distribution of the quantity $\ln(M/E)$, where $M$ is a random variable representing the \underline{M}odel prediction and $E$ is a random variable representing the \underline{E}xperimental measurement. Recall from Chapter~\ref{Error_Chapter} that $\ln(M/E)$ is assumed to be normally distributed. To test this assumption for each of the quantities of interest listed in Table~\ref{summary_stats}, Spiegelhalter's normality test has been applied~\cite{Spiegelhalter:Biometrika1983}. This test examines a set of values, $x_1,...,x_n$ whose mean and standard deviation are computed as follows:
\be
   \bar{x} = \sum_{i=1}^n x_i  \quad ; \quad \sigma^2 = \frac{1}{n-1}  \sum_{i=1}^n \left( x_i - \bar{x} \right)^2
\ee
Spiegelhalter tests the null hypothesis that the sample $x_i$ is taken from a normally distributed population. The test statistic, $S$, is defined:
\be
   S = \frac{N-0.73 \, n}{0.9 \, \sqrt{n}}  \quad ; \quad N=\sum_{i=1}^n Z_i^2 \, \ln \, Z_i^2  \quad ; \quad Z_i = \frac{x_i - \bar{x}}{\sigma}
\ee
Under the null hypothesis, the test statistic is normally distributed with mean 0 and standard deviation of 1. If the $p$-value
\be
   p = 1 - \left| \erf \left( \frac{S}{\sqrt{2}} \right) \right|
\ee
is less than 0.05, the null hypothesis is rejected.

The flaw in most normality tests is that they tend to reject the assumption of normality when the number of samples is relatively large. As can be seen in some of the histograms on the following pages, some fairly ``normal'' looking distributions fail while decidedly non-normal distributions pass. For this reason, the $p$-value is less important than the qualitative appearance of the histogram. If the histogram exhibits the typical bell-shaped curve, this adds confidence to the statistical treatment of the data. If the histogram is not bell-shaped, this might cast doubt on the statistical treatment for that particular quantity.

\IfFileExists{SCRIPT_FIGURES/ScatterPlots/validation_histograms.tex}{\input{SCRIPT_FIGURES/ScatterPlots/validation_histograms.tex}}{\typeout{Error: Missing file SCRIPT_FIGURES/ScatterPlots/validation_histograms.tex}}


\clearpage


\section{Summary of FDS Validation Git Statistics}

Table~\ref{validation_git_stats} shows the Git repository statistics for all of the validation datasets. For each dataset, the corresponding last changed date and Git revision string are shown. This indicates the Git revision string and date for which the most recent validation results for a given dataset were committed to the repository.

\IfFileExists{SCRIPT_FIGURES/ScatterPlots/validation_git_stats.tex}{\input{SCRIPT_FIGURES/ScatterPlots/validation_git_stats.tex}}{\typeout{Error: Missing file SCRIPT_FIGURES/ScatterPlots/validation_git_stats.tex}}


\bibliography{../Bibliography/FDS_refs,../Bibliography/FDS_general,../Bibliography/FDS_mathcomp}

\end{document}
